I can relate too. The time that feels wasted to me is the time we spent married but putting off children because we thought armageddon was coming. Eventually my body clock kicked me up the arse and we had children. I will say that after leaving I have an outlook which has enabled me to be pretty single-minded in making up for lost time and I don't feel hugely disadvantaged. In fact I look around and see some people who dropped out of university for other reasons and are now struggling to study part time like me.
I do feel for those who get to middle age before exiting. Like Ray Franz. Not being overtaken by bitterness must be a daily struggle.